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This paper examines proposed changes in a 
principal -preparation program in a medium- sized state university. The report 
discusses the implications of national and state standards for school leaders 
on program quality, describes the design ana initial implementation of a 
longitudinal study instituted to assess student perceptions of program value 
and usefulness, and seeks to generate a dialogue with colleagues interested 
in principal preparation. The article is based on a multiple-case study. In 
the pilot phase, data were gathered by asking students to complete a short 
survey providing demographic data and information about the value of their 
program. Then, selected students discussed their experiences in the program. 
Focus groups, student writings, program documents, and researcher reflection 
on program changes were also used to accumulate information. The study 
provided rich descriptive data regarding the experience of students, but 
these data were limited in that they could not be generalized to other 
programs- -the data identified specific student perspectives on the value of 
selected administrator-preparation programs but were limited by the students' 
uniqueness. Future studies should explore the appropriate balance between 
theoretical issues of school leadership and the practical learning necessary 
for success as a school leader. (Contains 46 references.) (RJM) 
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As demand for school leaders rises school districts report a significant 
shortfall in the pool of aspiring school leaders (Educational Research Service, 
1998). One report on the shortage of candidates suggested that the decline 
represents a recognition by potential applicants that the principals' job is more 
complex than ever before (Anderson, 1991). More recently a study by the 
Educational Research Service (ERS) (1998) affirmed the shortage. Nearly half 
of districts surveyed, regardless of location (rural, urban, suburban) or level 
(elementary, middle, high school), reported a shortfall. 

It is critical, therefore, that preparation programs be designed to meet both 
needs— prepare candidates in sufficient numbers to meet the demand, and 
ensure quality candidates ready to meet the challenges of the contemporary 
principalship. In North Carolina, a committee created by the General Assembly 
studied the problem and framed recommendations as follows: 

Recent changes in the public schools have placed increasing 
demands on public school administrators. Research on effective schools 
and the movement toward site-based decision making have identified the 
principal as a key factor in efforts to improve schools. It is imperative 
therefore that competent persons be attracted to and prepared for a career 
in school administration (Quality Candidate Committee, 1994). 

The concerns raised in North Carolina were representative of concerns 
expressed in other states. In Anderson's (1991) review of principal preparation 
programs, he claimed that practicing administrators have long been dissatisfied 
with their graduate training. According to Anderson, "the central problem 
appears to be that most university programs present knowledge about school 
administration, but do not help students develop skills to translate that 
knowledge into practice" (p. 6). This is particularly important because the 
principal is a key agent to achieve educational excellence. Given this belief, 
Anderson criticized traditional training programs for "not providing the field- 
based experiences necessary for developing outstanding principals" (p. 9). 

Similarly, the University Council for Educational Administration (1987) cited 
the lack of preparation programs relevant to the job related demands 
encountered by school administrators as one problem related to principal 
preparation. Concerns were also expressed by the National Policy Board for 
Educational Administration (1989) which called for a year long field residency to 
address the problems. The National Commission for the Principalship (1990) 
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also called for preparation programs that were based on the realities of the 
workplace. 

Nationally, there are calls for changes in principal preparation programs 
designed to increase their effectiveness in preparing new principals for 
tomorrow’s challenges (National Policy Board for Educational Administration, 
1989; National Commission for the Principalship, 1990). Existing and emerging 
national standards call for programs that are both inclusive of core knowledge 
and problem-based and grounded in practice (Council of Chief State School 
Officers, 1996). 

The purpose of this presentation is to discuss an investigation of the 
implications of these changes on one principal preparation program in a 
medium sized state university. Specifically, the purposes are: 

• To examine the implications of national and state standards for school 
leaders on program quality in one principal preparation program, 

• To share the design and initial implementation of a longitudinal study 
instituted to assess student perceptions of program value and usefulness, 
and 

• To generate discussion with and seek the advice of colleagues interested 
in principal preparation in order to enhance the research design of this 
study. 



Statement of the Problem 

The creation of standards and the emphasis on aligning preparation 
programs with practice are consistent with recommendations for improved 
administrator preparation programs. Jenkins and Bebar (1994) found that clear 
performance domains can positively impact preparation programs. Similarly, 
Thomson (1993) added that the absence of uniform licensure standards and a 
common framework of knowledge and skills negatively impacted the 
preparation of professional principals. The emphasis on practice is supported 
by researchers like Richardson and Lane (1994) who argued that it is essential 
that principals develop critical analysis skills and the ability to apply those skills 
in school settings. More specifically, Holifield and King (1993) identified a 
number of needs of aspiring and beginning principals that were clearly practice- 
based, including finance, law, staff relations, student discipline and motivation, 
scheduling, and issues of exceptional children. 
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In response to demands for greater accountability in the preparation of 
school leaders, North Carolina reconfigured its principal licensure process. In 
addition to completing a Masters of School Administration degree, aspiring 
principals must also successfully pass the School Leaders Licensure 
Assessment developed by Educational Testing Service (ETS) and designed to 
measure a student's capacity to meet the Interstate School Leaders Licensure 
Consortium (ISLLC) Standards. 

The introduction of state and national standards, complemented by the 
requirement to pass a national test based on those standards altered the 
context in which educational leadership programs function. Several important 
questions have emerged: 

• What are the implications of national standards on the content and 
pedagogy of preparation programs? 

• What role do preparation programs play in assisting students to 
successfully complete licensure requirements? 

• What factors do students use to assess quality preparation programs? 

• What strategies may be used by programs to prepare students for school 
leadership in a changing educational context? (Williamson & Hudson, 
1998b) 



Context of this Study 

Early in this decade, the North Carolina General Assembly launched an 
initiative to examine administrator preparation programs in the state. At the time 
there was tremendous variety among the programs and significant difference in 
the perceptions of quality by prospective administrators completing such 
programs and their employers (Quality Candidate Committee, 1994). At the 
time it was possible to be licensed as a school leader by taking a few classes 
and earning an endorsement to another masters degree. 

The General Assembly established an Educational Leadership Task Force 
which found that administrator training programs and entrance standards to 
those programs should be improved. Task Force members believed that 
strengthening leadership training would improve the quality of the state's public 
schools. 

As a result of the Task Force's recommendations, the state disestablished 
all "certification only" and master's level administrator preparation programs. To 
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replace these programs the state invited the sixteen campuses of the state 
university system to submit proposals for creation of a new Master's in School 
Administration program (MSA). This competitive proposal process resulted in a 
reduction in the number of Master's programs to seven (later nine) across the 
state. 

The request for proposals specified a number of criteria for judging the 
merits of the proposals. Among the more significant were an emphasis on a 
common core of knowledge and skills grounded in practice, the use of varied 
methods of instruction, and the integration of clinical components throughout 
the program. Another requirement of the new programs was that they provide 
students with a significant, active, full-time internship experience. 

New Masters of School Administration Programs 

The new Master's in School Administration programs were selected on 
several criteria including their ability to show how the common core of 
knowledge and skills emphasized in the program would be grounded in 
problems of practice. It was also necessary that schools demonstrate how 
instructional practice and methodology would incorporate practice and 
problem-based approaches. 

Principal Fellows Program 

As a result of the study of preparation programs for school leaders in North 
Carolina, referred to earlier, the General Assembly established the Principal 
Fellows Program. The program was created to ensure that the "best, most 
highly qualified students" are able to attend master's programs in school 
administration for two years on a full-time basis. These programs offered one 
year of full-time academic study, enrichment activities, and a one-year 
internship in a North Carolina public school. 

This university is one of nine state institutions participating in the Principal 
Fellows Program. During the first five years of the program 34 Principal Fellows 
have enrolled in the Masters of School Administration program. 

Standards for School Leaders 

In 1993, the state legislature created the Standards Board for Public 
School Administration and charged it with establishing standards for the 
licensure of administrators in North Carolina. The Standards Board, after years 
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of work and consultation with experts and practitioners across the state, created 
a set of ten standards identifying what the state's future school leaders should 
believe, know, and be able to do. Expected knowledge, skills, and professional 
perspectives were delineated for each of the ten standards. 

Both the national standards and those developed for North Carolina 
emphasize the complexity of the leadership role (Bolman & Deal, 1991), the 
importance of moral and ethical grounding (Kouzes & Posner, 1993; 
Sergiovanni, 1992; 1996), the value of working closely with parents and 
community (Epstein, 1993; Prestine, 1991; Sergiovanni, 1994), and the 
importance of student learning as the primary function of schools (Newman, 
1991). 

The national standards identify teaching and learning as the primary 
purpose of schooling (Council of Chief State School Officers, 1996) and this 
emerged as the central theme. Complementing the focus on learning and 
student achievement is the importance of assuring that schools consider the 
individual needs of students. Candidates, therefore, must examine each 
scenario on the licensure test in light of its impact on the student— often requiring 
deviation from established practice and policy. 

The importance of working with parents and community to support student 
learning is yet another theme. Establishing a close working relationship with 
parents and designing ways in which parents and other care givers can be 
actively involved in school life is central to contemporary school leadership 
(Comer 1996, Epstein, 1993). 

Ten performance domains were identified in the North Carolina standards 
(North Carolina Standards Board, 1998). While greater in number and of 
greater specificity in some areas, the North Carolina standards parallel the six 
ISLLC standards for school leaders. Student achievement and success is the 
foundation upon which the other standards rest. The school leader is 
envisioned as one who understands the importance of working with teachers, 
parents, and community to establish a successful learning environment for 
students. 

Licensure Exam 

Accompanying the adoption of state and national standards, a consortium 
of states (District of Columbia, Illinois, Kentucky, Mississippi, Missouri, and 
North Carolina) contracted with Educational Testing Service (ETS) to develop a 
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licensure assessment. Known as the Interstate School Leaders Licensure 
Consortium (ISLLC), this group, under the auspices of the Council of Chief 
State School Officers, merged their state standards into six ISLLC standards 
and worked with Educational Testing Service to develop a practice-grounded, 
problem-based examination that would assist states in determining that 
prospective administrators were "safe to practice"--"a mechanism that helps 
ensure that only individuals who possess important knowledge and skills enter 
into professional practice" (Educational Testing Service, 1997, p. 6). 

This determination would be based on scores on the School Leaders 
Licensure Assessment, scored using rubrics closely aligned to the standards. 

In other words, students would be expected to answer practice-and problem- 
based questions in ways which demonstrated that their knowledge, 
dispositions, and performances were consistent with the standards. Initiated in 
January 1998, this test is now used to determine who is licensed to practice as 
a school administrator in North Carolina. Student scores will also be used as 
one measure of the success of the Master's in School Administration program at 
each of the participating schools. 

Varying from traditional paper and pencil tests the School Leaders 
Licensure Assessment asks students to respond to a series of authentic tasks 
each designed to elicit the students' thinking and approaches to resolving 
complex and occasionally contentious leadership issues. Composed of four 
sections, the six-hour test expects students to use documents and data from real 
school-life situations, and critically examine the strategies used by other school 
leaders on a range of situations. 

Methodology 

This research, based on activities of the educational leadership program at 
one state university, explores the impact of national standards for school 
leaders on the structure, content, and quality of the program. 

A multiple case study approach was selected for gathering these data 
(Ashton & Webb, 1986; Lightfoot, 1983; McLaughlin, Irby & Langman, 1994; 
Polakow, 1993) in order to examine the multiple perspectives that students 
bring to the program. In the pilot phase, data were gathered in two ways 
(Williamson & Hudson, 1998a). First, students completed a short survey 
providing demographic data and information about the value of the program. 
Second, selected students were invited to meet with one of the researchers and 
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participate in interviews about their experience in the MSA program. Each 
interview was taped and later transcribed and analyzed for commonalities in 
language, themes, and perceptions. Neither rigid adherence to an interview 
guide nor forced respondent compliance was utilized. Priority was given to the 
dynamic and spontaneous nature of each interview and to the development of a 
trusting relationship between respondent and researcher (Yin, 1994). 

In the second phase of the study analysis of student writing was added to 
surveys and interviews to enhance understanding of student perspectives. In 
addition, students were asked to complete a Critical Incident Report describing 
a significant event which molded their thinking about school leadership. At the 
end of the first year of employment, employers of selected graduates were 
interviewed regarding the student's preparation for school leadership and ways 
in which the university might refine its preparation program. 

Data Sources 

The data sources used in this study are primary and naturalistic in nature. 
Primary data sources for the pilot phase of this study were student surveys, 
individual and focus group interviews, and program documents. During the 
second phase of the study, sources were student surveys, interviews, focus 
groups, student writings, program documents, and researcher reflection on 
program changes. 

A variety of methods may be used for data collection in a case study. Yin 
(1994) identified six different sources of information — documents, archival 
records, interviews, direct observation, participant-observation, and physical 
artifacts. Documents include letters, memoranda, agendas, and other formal 
written reports or communiques. Archival records consist of survey data, 
personal records such as diaries and telephone contacts, organizational charts, 
and budgets (Yin, 1994). 

The second source of information suggested for case study information is 
the interview, considered one of the most important sources of information for 
case studies (Spradley, 1979; Yin, 1994). Three types of interviews are often 
included in case study methodology: open-ended, focused, and structured. 
Open-ended interviews provide the opportunity for greater interaction with the 
respondent and allow respondents to enunciate their own insights into the issue 
being investigated (Spradley, 1979). 
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Focused interviews serve an important function in case study research. 

They provide an opportunity for the researcher to corroborate certain facts 
which have already emerged from the documentation. The role of the 
researcher is to ask specific questions which, when carefully worded, invite 
respondents to provide their own new or unique perspective on the topic (Yin, 
1994). 

A third data collection method utilized in case study research is direct 
observation. Using this method the investigator visits the site and observes the 
respondents. Such a strategy may involve formal observation of meetings or 
classrooms and less formal observation such as those made when conducting 
interviews or other data collecting strategies. 

Participant observation is the fourth method of case study data collection 
suggested by Yin (1994). It is a "special mode of observation in which the 
investigator is not merely a passive observer. Instead, the investigator may take 
a variety of roles within a case study situation and may actually participate in the 
events being studied" (p. 92). 

The final method of case study data collection is the use of physical 
artifacts( Yin, 1994, p. 94). Such artifacts may include a tool, a work of art, or 
some other physical evidence. 

This study used several data collection methods. They included: a student 
survey and survey of employers, focused interviews with students and 
employers, a critical incident report to provoke thinking about significant events 
during a student's preparation program, collection and analysis of student 
writing, a group interview of students both during the program and following 
graduation. 

Collection of Data 

Each element of the study utilized a different data collection method. Table I 
outlines the data sources and collection schedule for this investigation using the 
crosswalk technique (O'Sullivan, 1990). 

The first phase, the pilot, involved data compiled from a student survey 
administered in the month prior to graduation in May 1998. During this same 
time selected groups of students participated in focused interviews in which 
their perceptions of the quality of the program and their preparation were 
elicited. 
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Based on the experiences gained from the pilot study (Williamson & 

Hudson, 1998a) data collection was refined for the formal study and consisted 
of data from a student survey administered on entry to the program, an analysis 
of student writing about the reasons they aspire to school leadership, and a 
review of program documents. 

Subsequent phases of the study will continue to gather data from these 
sources. Additional data will be collected from student mentors during their 
internship, from exit surveys upon graduation, from employers during their initial 
employment as a school leader, and from placement data maintained by the 
School of Education. 

An eight-step data collection process was utilized for this study. Each step 
was designed to gather information about student attitudes toward school 
leadership and perceptions of quality in their leadership preparation program. 

Phase 1 - The Pilot Study. Initial data gathering for phase one was comprised 
of two separate activities (Williamson & Hudson, 1998a). 

Step 1 : Demographic Survey : Students completed a short survey 
constructed to identify underlying demographic variables which might impact 
the findings of this investigation. Students enrolled in the final semester of the 
MSA program were asked to respond to the questions. Based on that 
information a demographic profile of students emerged. 

Step 2: Focused Interviews . Following examination of this data selected 
students were invited to meet with one of the researchers and participate in a 
lengthy focused interview about their experience in the MSA program. Each 
interview was taped and later transcribed and analyzed for commonalities in 
language, themes, and perceptions of the MSA program. 

Open-ended questions were asked based upon the information provided in 
the surveys. The interviews allowed the researchers to probe the written 
responses and elicit information from the respondents which would elaborate 
on their thinking. Information provided by the respondents was redirected to the 
respondents for clarification and explication. Neither rigid adherence to an 
interview guide nor forced respondent compliance was utilized. Priority was 
given to the dynamic and spontaneous nature of each interview and to the 
development of a trusting relationship between respondent and researcher 
(Yin, 1994). 
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Table 1 

Evaluation Cross-Walk 



Questions 
Pilot Study 

1 . Who are our students? 

2. How do students describe their 
experience in the MSA program? 

3. What elements of the MSA program 
do students find most useful? of 
greatest value? 

Formal Study 

1 . What are the demographics of 
students in the MSA program? 

2 What do students believe about 
leadership when they begin their 
preparation program? 

3. How do student beliefs about 
leadership change during their time 
in the MSA program? 

4. What critical events mold and shape 
student thinking about leadership? 

5. What factors do students consider in 
selecting a preparation program? 

6. What elements of the MSA program 
do students find most useful? of 
greatest value? 

7. How successful are our graduates 
when they become school leaders? 



Data Sources 
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The focus of this investigation was to identify those factors which motivated 
students to enter the program, and to elicit student perceptions about the quality 
of program components including their classroom and internship experiences. 

Phase 2 - The Formal Study. Several sources of data were used during this 
part of the study. 

Step 3: Student Survey - Each August a new cohort of students enters the 
MSA program. Based on the pilot study the researchers refined the student 
survey and interview protocols. The refined survey was completed by all 
students entering the program in August 1998 (Williamson & Hudson, 1998c). 
The survey provided a detailed demographic profile of students in the program 
and captured, in their own words, initial beliefs about leadership and quality 
preparation programs. Similar data will be gathered at several key junctures 
throughout the students' time in the program and upon program completion. 

Step 4: Student Written Work . During the first semester in the program 
students were asked to prepare a written statement of beliefs about leadership 
and the role of the school leader. Upon completion of the program students will 
prepare a similar statement for their exit portfolio. Both writing samples will be 
reviewed to identify key words and phrases which illustrate student thinking 
about leadership. The documents will be analyzed to identify patterns and 
trends in student thinking during their time in the program. 

Step 5: Focused Interviews . Selected students were invited to meet with 
one of the researchers and participate in a lengthy focused interview about their 
experience in the MSA program. The interviews were taped and later 
transcribed and analyzed to identify patterns in language, themes, and 
perceptions of the MSA program. 

As during the pilot study open-ended questions were used to probe written 
responses from the survye and elicit information from the respondents which 
would elaborate on their thinking. Similar strategies were utilized: information 
provided by respondents was redirected for clarification and explication, rigid 
adherence to an interview guide and forced respondent compliance was 
avoided. In each case priority was given development of a trusting relationship 
between respondent and researcher (Yin, 1994). 
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These interviews asked students to identify those factors which motivated 
entry the program, and to talk about their expereience in the program and their 
perceptions of those program elements of greatest value (Hudson & Williamson, 
1999). 

Step 6: Critical Incident Reports . All students must complete a year-long 
internship as part of the Masters of School Administration program. During this 
internship students will be asked to complete a Critical Incident Report. The 
report was designed to heighten student awareness of an incident which they 
experienced which impacted their perceptions of the MSA program. Each 
student will have full authority for selection of the incident. The only guidelines 
provided to respondents will be that the incident have special significance to 
them and that it served as a catalyst for clarifying and understanding the role of 
the preparation program in their development as a school leader. 

Step 7: Employer Interviews . During the first year of employment as a 
school leader the student's employer will be invited to participate in a focused 
interview about the quality of the student's preparation for school leadership 
and their beliefs about strategies which the university might use to refine and 
strengthen its preparation program. These interviews will be taped, 
transcribed, and the data analyzed to ascertain trends and patterns. 

Step 8: Documents and Other Artifacts . Each student will be invited to 
provide documents and other artifacts which reflect on their preparation for 
school leadership. Additionally, documents from the School of Education, 
including student test results and placement information will be gathered and 
analyzed to identify trends and patterns. 

Analysis of Data 

The data collection methods established for this study provided an array of 
statements, documents, and observations. All information was organized, 
categorized, analyzed, and synthesized beginning with initial data collection as 
suggested by Fetterman (1989), Glesne and Peshkin (1992), Miles and 
Huberman (1994), and Yin (1994). Glesne and Peshkin (1992) noted that "data 
analysis done simultaneously with data collection enables you to focus and 
shape the study as it proceeds" (p. 127). 



Following transcription of the interviews, the transcripts were analyzed by 
the researchers and a tally of key words or phrases was obtained. This analysis 
assisted the researchers in focusing subsequent data analysis activities on 
these descriptors. 

Several strategies were suggested for analysis of data during a case study. 
They included writing memos to oneself or keeping a reflective field log (Glaser 
& Strauss, 1967); preparation of analytic files organized by generic category 
such as title, introduction, conclusion, quotations (Lofland, 1971); and use of 
coding systems to organize information (Miles & Huberman, 1994). 

Charles (1995) suggested a procedure for analyzing case study 
information. It involved a four-step process: "(1) identification of topics, (2) 
clustering of topics into categories, (3) forming the categories into patterns, and 
(4) making explanations from what the patterns suggest" (p. 121). 

Three approaches to qualitative data analysis were suggested by Miles and 
Huberman (1994). First, was data reduction consisting of "selecting, simplifying, 
abstracting, and transforming" the data (p. 10). The authors suggested that data 
reduction is a part of data analysis that "sharpens, sorts, focuses, discards, and 
organizes data in such a way that final conclusions can be drawn and verified" 

(p. 11). 

The second data analysis consideration suggested by Miles and Huberman 
(1994) was the need for data display. This consists of an organized 
presentation of information. The authors suggested that such a display often 
consists of extended text or the use of graphics including matrices, graphs, and 
charts. Construction of such displays was part of analysis and assisted in 
developing and drawing conclusions. 

The third recommended analysis activity was conclusion drawing and 
verification. This activity consists of "noting regularities, patterns, explanations, 
possible configurations, causal flows, and propositions" (Miles & Huberman, 
1994, p. 11). Verification consists of using the emerging data set to find 
corroborating information for the conclusions. It includes testing meanings "for 
their plausibility, their sturdiness, their confirmability-that is, their validity" (p. 

11 ). 

An ongoing data analysis process was utilized for this study (Eisner, 1991; 
Yin, 1994). Information was arranged in files for each cohort of students (Glesne 
& Peshkin (1992). Sources of information were charted and coded (Miles & 
Huberman, 1994). Charles' (1995) four steps were utilized to identify topics, 



- 15 - 



cluster topics into categories, form categories into patterns, and develop 
conclusions based upon the patterns. 

The analysis of data followed five specific steps. The following section 
describes each of those steps and illustrates its application to this investigation. 

1 . After each student interview the tape of the interview was transcribed and 
a written record prepared using a commercially available word 
processing program. This created a computer file for each interview. This 
activity provided an opportunity to identify key words which were used 
repeatedly throughout the conversations. Transcribing each tape also 
provided an occasion to listen to each interview again. 

2. Following review of each transcript the highlighted segments were read, 
clustered into categories and assigned titles. Each subsequent interview 
was handled the same way. The highlighted sections of each interview 
were compared and clustered into categories. Occasionally a new 
category emerged. 

3. The categories which arose from this review were examined and 
grouped into meaningful patterns. The frequency of statements using key 
words or phrases was tabulated and examined. This resulted in 
regrouping and occasionally discarding categories. 

4. Distinct patterns emerged from this review. Each pattern was studied to 
establish commonalities in the information, uniqueness of each event, 
and missing information which might be gathered from further review of 
the transcripts and documents. Particular attention was given to 
identification of dissonant voices (Lawrence-Lightfoot & Davis, 1997). 

5. Several themes surfaced in this study. After delineating the themes a 
new set of transcripts was printed and used to identify statements by 
participants which illustrated each theme. Different colored markers were 
used, one for each theme, to highlight items in each transcript. A note 
pad on which the critical elements of each theme were outlined was 
prepared. The transcript pages containing statements and examples 
illustrating each theme were noted. These note pages were used to 
report the data from this investigation. 
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Validitv and Reliability 

Four tests are suggested for judging the quality of research designs (Kidder, 
1981). They include construct validity, internal validity, external validity, and 
reliability. 

Construct validity is defined as "establishing correct operational measures 
for the concepts being studied" (Kidder, 1981, p. 7). Yin (1994) suggested three 
tactics for case-study research which address this test. They included use of 
multiple sources of evidence, establishing a chain of evidence, and having key 
informants review drafts of the case study report. 

All three approaches to establishing validity were used for this study. 

Multiple sources of information were gathered from surveys, focused interviews, 
review of documents and other artifacts, and Critical Incident Reports. Each data 
source was charted and tracked. A coding schema permitted the researchers to 
establish a chain of evidence. Finally, upon completion of preliminary findings 
the researchers asked respondents to review the findings and comment on their 
appropriateness. 

The second criterion for judging the quality of research design is internal 
validity (Kidder, 1981). This criterion involves establishing a causal relationship 
showing that certain conditions lead to other conditions. Yin (1994) suggested 
that pattern matching and explanation-building are two strategies which assure 
internal validity of case study research design. 

Explanation building is suggested as a test of internal validity (Yin, 1994). 
This is described as "narrative building" (p. 1 14) and involves telling the story of 
the various cases. It involves describing the events at each site, providing a 
general explanation, comparing individual cases to the explanation, and 
refining the explanation on the basis of each new case. 

Use of explanation building as a strategy for data presentation and analysis 
was used for this investigation. Such a strategy established the chain of 
evidence as a mechanism for presentation and explanation of the students 
evolving views of leadership and of their preparation programs. 

External Validity 

The third test of the quality of a research design involves "knowing whether 
a study's findings are generalizable beyond the immediate case study" (Yin, 
1994, p. 43). 




17 



- 17 - 



This study investigated the impact of a school leadership preparation 
program on student perceptions about the role of school leaders. The study 
provided rich descriptive data regarding the experience of students. However, 
these data are specifically limited in that they cannot be generalized to other 
cases. 

Reliability 

The fourth test of the quality of a research design is reliability (Kidder, 

1981). It is defined as "demonstrating that the operations of a study-such as 
data collection procedures— can be repeated, with the same results" (p. 8). Miles 
and Huberman (1994) described the reliability issue as "whether the process of 
the study is consistent, reasonably stable over time and across researchers and 
methods . . . Have things been done with reasonable care?" (p. 278). 

Yin (1994) offered suggestions for addressing the reliability question. First 
is the development of a detailed description of the procedures used throughout 
the case study. This may include the development of a case study protocol and 
a case study data base. Second, is to make as many steps as possible 
operational. Yin described it as conducting "research as if someone were 
always looking over your shoulder" (p. 45). 

A protocol was developed for this case study. The protocol included the 
steps utilized in examining student data, the process for developing interview 
guides for focused interviews, and a system for coding and analyzing the 
information generated by the data collection instruments. 

The data for this study was gathered from students in the natural setting of 
the MSA program. What Lincoln and Guba (1985) call naturalistic inquiry, 
others call a phenomenological approach. Borg, Gall and Gall (1993) 
elaborated on the value of such an approach. It allows the researcher to 
"develop an understanding of individuals and events in their natural state, 
taking into account the relevant context" (p. 194). It is based on an appreciation 
for the uniqueness of each individual and the settings in which they live and 
work. This "phenomenological reality" (p. 194) is particularly relevant when the 
researchers want to examine and understand a program or event from "the 
perspective of the participants" (p. 195). 

While such studies provide valuable insights into the thinking of the 
subjects, they are limited in the ability to make generalizations based on their 
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findings. Findings are very context-specific, reflecting the unique orientation of 
the subjects. Therefore, this study is limited in two ways: 

• The information generated by this study, while useful in identifying specific 
student perspectives on the value of selected administrator preparation 
programs, is limited by the unique characteristics of the students who 
participated in the study and the program in which they were enrolled. 

• The results of the study are not generalizable and cannot be construed to 
be applicable to other programs in other locations. 

Questions for Further Study 

As a result of the pilot and initial data collection several questions emerged 
which will guide subsequent phases of this investigation. They include: 

• What is the appropriate balance between examination of larger, more 
theoretical issues of school leadership and the practical learnings 
necessary for success as school leaders? 

• How do preparation programs assure student success on licensure 
examinations and at the same time provide students with the opportunity 
to "stretch." to "talk about issues not talked about in schools," and to 
"expand their horizons?" 

• What are the implications of national standards on content and pedagogy, 
on the balance between theory and practice? 

• What exactly is the role of preparation programs in preparing students to 
work in diverse settings? 

• How do preparation programs provide for learning that is meaningful, 
authentic, and immediately applicable to problems of practice? 

• How does a program ensure that case studies, problem-based learning 
activities, vignettes, data analyses, and other such practice based 
activities are an integral part of all course work, regardless of the teacher? 
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